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Advancement  Degree  of  Difficulty  (AD2)  is  a  method  of 
systematically  dealing  with  aspects  beyond  TRL. 

It  is  a  “predictive”  description  of  what  is  required  to  move  a 
system,  subsystem  or  component  from  one  TRL  to  another. 

It  provides  information  in  the  form  of: 

•  Liklihood  of  occurrence  of  an  adverse  event.  >  Risk 

•  Cost  to  ensure  that  such  an  event  does  not  occur,  Impact 

•  The  time  required  to  implement  the  necessary  action^/ 


Using  AD2  as  an  Input 
to  Risk  Management 
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•  AD2  consists  of  a  set  of  questions  in  5  specific  areas: 

-  Design  and  Analysis 

-  Manufacturing 

-  Software  Development 

-  Test 


-  Operations 

•  The  questions  are  asked  about  each  element  in  the  product 
WBS  structure  from  the  top  level  system  down  to  the 
individual  component. 

•  The  questions  are  not  directed  toward  the  element  itself,  rather 
toward  the  issue  of: 


-  Do  you  have  the  resources  -  people,  skills,  tools,  facilities,  etc.  to 
design,  manufacture,  test  and  operate  it? 


Using  AD2  as  an  Input 
to  Risk  Management 
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The  levels  of  risk  associated  with  AD2  are  described 
in  terms  of  the  experience  base  of  the  developers. 

i.e.,  have  they  done  this  before? 
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Requires  new  development  outside  of  any  existing  experience  base.  No 
viable  approaches  exist  that  can  be  pursued  with  any  degree  of  confidence. 
Basic  research  in  key  areas  needed  before  feasible  approaches  can  be 
defined. _ 

Requires  new  development  where  similarity  to  existing  experience  base 
can  be  defined  only  in  the  broadest  sense.  Multiple  development  routes 
must  be  pursued. _ 

Requires  new  development  but  similarity  to  existing  experience  is 
sufficient  to  warrant  comparison  in  only  a  subset  of  critical  areas. 

Multiple  development  routes  must  be  pursued. 

Requires  new  development  but  similarity  to  existing  experience  is 
sufficient  to  warrant  comparison  on  only  a  subset  of  critical  areas.  Dual 
development  approaches  should  be  pursued  in  order  to  achieve  a  moderate 
degree  of  confidence  for  success,  (desired  performance  can  be  achieved  in 
subsequent  block  upgrades  with  high  degree  of  confidence. _ 

Requires  new  development  but  similarity  to  existing  experience  is 
sufficient  to  warrant  comparison  in  all  critical  areas.  Dual  development 
approaches  should  be  pursued  to  provide  a  high  degree  of  confidence  for 
success. _ 

Requires  new  development  but  similarity  to  existing  experience  is 
sufficient  to  warrant  comparison  across  the  board.  A  single  development 
approach  can  be  taken  with  a  high  degree  of  confidence  for  success. _ 

Requires  new  development  well  within  the  experience  base.  A  single 
development  approach  is  adequate. _ 

Exists  but  requires  major  modifications.  A  single  development  approach 
is  adequate. _ 

Exists  with  no  or  only  minor  modifications  being  required.  A  single 
development  approach  is  adequate. _ 
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Advancement  Degree  of  Difficulty  -  Questions 


Save  It 


Close 

Calculator 


AD2  Start 


Create  Summary 

View  Degree  of 

Index  of  AD2 

Index  of  Saved 

Of  Results 

Difficulty  Criteria 

Projects 

Records 

If  you  wish  to  add  more  questions,  uncheckl 

hnv  at  rinht  arid  wonr  nnpctinnf  thpn  rprhprk | 

WBS  Product  Hierarchy  Name 


Hide  Blank  Questions?  □ 


Today's  Date: 

Project: 
Title: 
Evaluator: 
Evaluation  Date  (Saved  data  only): 


8/22/2008 


Example 


Air  Tank  Bleed  Valve  2 


J.  Cole 


WBS# 


System/Subsystem 

Subsystem/Componer 


Pressure  control 

al.2.3.5 

2nd  Bleed  valve 

al.2.3.5.22 

The  additional  level  can  be  used  to 
provide  more  depth  to  the  assessment. 


AD2WBS 

Roll-Up 


2/22/08  9:44  AM 


N.B.  The  name  of  the  "Title"  is  used  to 
identify  saved  data. 


Clear  Entries 


Change  Schedule  &  Cost  Ranges 


AD2  Criteria  ffWSFl  IWWKI  IWHFl  flSWH  IWHH  IISWW  IfSWFI  AD2  Criteria 

Schedule  Cost  AD2  Level  Only  Answer  Questions  That  Apply 


New  Evaluation 
(Same  Project) 


Start  a  New 
Project 


Schedule 


"3 

"3 


"3 

"3 


t  to  3yr  ▼  | 

t  to  2yr  ▼  | 


"3 

"3 


0  to  6mo  ▼  | 

2yr  to  3yr ▼] 


Comments 


Questions 


Cost  AD2  Level 

ost ▼] 


■3 

3 


$10M  to  S20M ▼] 

S20M  to  S50M ▼] 


3 

3 


31 


S50M  to  S100M  ▼  | 

>  $100M ▼) 


Level  5:  40%  Risk  ' 


3 

Level  7:  60%  Risk  ▼  | 


Level  1:  0%  Risk 


3 


Level  5:  40%  Risk  ' 


3 

Need  more  data  ▼  | 


Level  7:  60%  Risk  ' 


3 

Level  3:  20%  Risk  ▼  | 


it  Applicable  ▼  | 


Level  5:  40%  Risk 


Level  5:  40%  Risk 


3 

3 


Do  the  necessary  data  bases  exist  and  if  not,  what  level  of 
development  is  required  to  produce  them? 

aaaaaaaaaaaaaaaaaaaa 

Do  the  necessary  desian  methods  exist  and  if  not.  what 
level  of  development  is  required  to  produce  them? 

bbbbbbbbbbbbbbbbbbbb 

Do  the  necessary  desiqn  tools  exist  and  if  not,  what  level 
of  development  is  required  to  produce  them? 

cccccccccccccccccc 

Do  the  necessary  analytical  methods  exist  and  if  not,  what 
level  of  development  is  required  to  produce  them? 

ddddddddd  ddddddd  dddddd 

Do  the  necessary  analysis  tools  exist  and  if  not.  what  level 
of  development  is  required  to  produce  them? 

eeeeeeee  eeeeeeee  eeeeeeeeee 

Do  the  appropriate  models  with  sufficient  accuracy  exist 
and  if  not,  what  level  of  development  is  required  to 
produce  them? 

fffffffffff 

Do  the  available  personnel  have  the  appropriate  skills  and 
if  not,  what  level  of  development  is  required  to  acquire 
them? 

qqqqgqgqqgqq 

Has  the  desian  been  optimized  for  manufacturability  and  if 
not,  what  level  of  development  is  required  to  optimize  it? 

hhhhhhhhhhhh 

Has  the  desian  been  optimized  for  testability  and  if  not. 
what  level  of  development  is  required  to  optimize  it? 

iiiiiiiiiiiiini 

Has  the  desian  been  optimized  for  intearation  at  the 
component,  subsystem  and  system  level  and  if  not,  what  is 
required  to  optimize  it? 

iiimiiiiiiiiii 
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Return  To 
AD2  Start 


AD2  Roll-up  of  Subsystem  Drivers 


Re-Calculate 
WBS  Roll-up 


Project:  Example 


Sensitivity  |  Level  7:  60%  Risk  ▼  | 


WBS 

Record 

Sub  Sys 

Comp 

Name 

5 

1.1.0 

5 

1.1.0 

Inducer 

3 

1.2.0 

Impeller 

4 

1.3.0 

1.3.1 

Pump  Housing 

4 

1.3.1 

Volute 

6 

1.3.2 

Diffuser 

7 

1.4.0 

Turbine  Blades 

8 

1.5.0 

Turbine  Nozzles 

11 

1.6.0 

1.6.1 

Turbine  Housing 

11 

1.6.1 

Manifolds 

9 

1.6.2 

Guide  Vanes 

10 

1.7.0 

Dynamic  Seals 

12 

1.8.0 

Bearings/Rotor 

13 

1.10.0 

Axial  Thrust  Balance 

14 

1.10.2 

Axial  Thrust  Balance2 

2 

al.2.3.5 

al.2.3.5.21 

Pressure  control 

2 

al.2.3.5.21 

Bleed  valve 

al.2.3.5.22 


2nd  Bleed  valve 


AD2  Current 
Evaluation 


Index  of  Saved 
Records 


Problem  Areas 


Schedule  Cost 


8/22/08  4:28  PM 


Index  of  AD2 
Projects 


Tech  Dev  Needed 


D&A  -  Necessary  data  bases 

zero  time 

zero  cost 

Level  7:  60%  Risk 

D&A  -  Appropriate  skills 

zero  time 

$50M  to$100M 

Level  7:  60%  Risk 

D&A  - 

zero  time 

zero  cost 

Level  8:  80%  Risk 

Mfg  -  Necessary  metrology 

zero  time 

$20M  to  $50M 

Level  7:  60%  Risk 

Mfg  -  Appropriate  skills 

0  to  6mo 

>$100M 

Level  7:  60%  Risk 

Mfg  - 

6mo  to  lyr 

$1M  to  $10M 

Level  7:  60%  Risk 

SW  Dev  - 

lyr  to  2yr 

$20M  to  $50M 

Level  7:  60%  Risk 

T&V  -  Test  facilities 

6mo  to  lyr 

$1M  to  $10M 

Level  7:  60%  Risk 

D&A  -  Necessary  design  methods 

zero  time 

$1M  to  $10M 

Level  7:  60%  Risk 

D&A  -  Necessary  analysis  tools 

2yr  to  3yr 

$20M  to  $50M 

Need  more  data 

D&A  -  Models  with  sufficient  accuracy 

lyr  to  2yr 

>$100M 

Level  7:  60%  Risk 

D&A  -  Optimized  for  manufacturability 

zero  time 

zero  cost 

Not  Applicable 

D&A  - 

zero  time 

zero  cost 

Level  7:  60%  Risk 

D&A  - 

2yr  to  3yr 

$50M  to$100M 

Level  9:  100%  Risk 

Mfg  -  Necessary  materials 

lyr  to  2yr 

$10M  to  $20M 

Need  more  data 

Mfg  -  Necessary  mfg.  tooling 

6mo  to  lyr 

$20M  to  $50M 

Not  Applicable 

Mfg  -  Necessary  metrology 

zero  time 

$20M  to  $50M 

Level  7:  60%  Risk 

Mfg  -  Necessary  mfg.  software 

0  to  6mo 

0to$1M 

Level  7:  60%  Risk 

Mfg  -  Brassboards 

zero  time 

zero  cost 

Not  Applicable 

Mfg  -  Qualification  models 

0  to  6mo 

$50M  to$100M 

Not  Applicable 

Mfg  - 

2yr  to  3yr 

0to$1M 

Need  more  data 

Mfg  - 

6mo  to  lyr 

$1M  to  $10M 

Level  9:  100%  Risk 

SW  Dev  - 

0  to  6mo 

$20M  to  $50M 

Level  8:  80%  Risk 

SW  Dev  - 

zero  time 

S20M  to  S50M 

Level  9:  100%  Risk 

SW  Dev  - 

zero  time 

$50M  to$100M 

Not  Applicable 

SW  Dev  - 

lyr  to  2yr 

$20M  to  $50M 

Need  more  data 

iternational 


Using  AD2  as  an  Input 
to  Risk  Management 


iSuliternational 


Relating  AD2  to  Project  Uncertainty:  from  Variation  to  Chaos* 

Variation: 

Cost,  time  and  performance  levels  vary  randomly,  but  in  a  predictable 
range. 


Foreseen  Certainty: 

A  few  known  factors  will  influence  the  project  but  in  predictable  ways. 


Unforeseen  Uncertainty: 

One  or  more  major  influence  factors  cannot  be  predicted. 


Chaos: 

Unforeseen  events  completely  dominate  the  project’s  target,  planning 
and  approach. 


*De  Meyer,  et  al 
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Project  Type 


Basic  Research 


Applied  Research 


Advanced  Research 


Advanced  Tech  Demonstrator 


Acquisition  Progra 


Project  Status 


Acceptable 


Reset  All 


TRL 


Adv.  Tech  Demo 


Level 


AD2 


Actual  system  completed  and  flight  qualified 
through  test  and  demonstration 


System/subsystem  model  or  prototype 
demonstration  in  a  relevant  environment 


Actual  system  flight  proven  through  successful 
mission  operations 


A 


System,  subsystem  model  or  prototype 
demonstration  in  a  relevant  environment 


Component  or  breadboard  validation  in  a  relevant 
environment 


Acceptable 


Component  or  breadboard  validation  in 
laboratory 


Analytical  and;  or  experimental  critical  function 
or  characteristic  proof-of-concept 


Technology  concept  or  application  formulated 


Basic  principles  observed  and  reported 


©X) 


Risk 


[Requires  new  development  outside  of  any  existing 

1  experience  base.  No  viable  approaches  exist  that  can  be 

1  pursued  with  any  degree  of  confidence.  Basic  research  in 
Ikey  areas  needed  before  feasible  approaches  can  be 

1  defined. 

IRequires  new  development  where  similarity  to  existing 

1  experience  base  can  be  defined  only  in  the  broadest  sense. 
[Multiple  development  routes  must  be  pursued. 

IRequires  new  development  bnt  similarity  to  existing 

1  experience  is  sufficient  to  warrant  comparison  in  only  a 

1  subset  of  critical  areas.  Multiple  development  routes  must 
[be  pursued. 

IRequires  new  development  but  similarity  to  existing 

1  experience  is  sufficient  to  warrant  comparison  on  only  a 

1  subset  of  critical  areas.  Dual  development  approaches 

1  should  be  pursued  in  order  to  achieve  a  moderate  degree 

1  of  confidence  for  success,  (desired  performance  can  be 

1  achieved  in  subsequent  block  upgrades  with  high  degree  of 
|  confidence. 

Requires  new  development  bnt  similarity  to  existing 
experience  is  sufficient  to  warrant  comparison  in  all  critical 
areas.  Dual  development  approaches  should  be  pursued  to 
provide  a  high  degree  of  confidence  for  success. 

40% 

IRequires  new  development  but  similarity  to  existing 
experience  is  sufficient  to  warrant  comparison  across  the 
board.  A  single  development  approach  can  be  taken  with  a 
high  degree  of  confidence  for  success. 

30% 

Requires  new  development  well  within  the  experience 
base.  A  single  development  approach  is  adequate. 

20% 

Exists  bnt  requires  major  modifications.  A  single 
development  approach  is  adequate. 

10% 

Exists  with  no  or  only  minor  modifications  being  required. 

A  single  development  approach  is  adequate. 

0% 

Likelihood 
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Relating  AD2  to  a  5X5  Risk  Matrix 


DOD  Likelihood  Descriptions 


Level 

Likelihood 

Probability  of  Occurrence 

1 

Not  Likely 

-10% 

2 

Low  Likelihood 

-30% 

3 

Likely 

-50% 

4 

Highly  Likely 

-70% 

5 

Near  Certainty 

-90% 

Consequence 
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Level 

Technical 

Schedule 

Schedule 

1 

Minimal  or  no  consequence  to 
technical  performance 

Minimal  or  no  impact 

Minimal  or  no  impact 

2 

Minor  reduction  in  technical 
performance  or  supportability,  can 
be  tolerated  with  little  or  no  impact 
on  the  program 

Able  to  meet  key  dates. 

Slip  <*month(s) 

Budget  increase  or  unit 
production  cost 
increases. 

<**(1%  of  Budget) 

3 

Moderate  reduction  in  technical 
performance  or  supportability  with 
limited  impact  on  program 
objectives 

Minor  schedule  slip.  Able  to  meet 
key  milestones  with  no  schedule  float 

Slip  <*month(s) 

Sub-system  slip<*month(s)  plus 
available  float 

Budget  increase  or  unit 
production  cost 
increases. 

<**(5%  of  Budget) 

4 

Significant  degradation  in  technical 
performance  or  major  shortfall  in 
supportability;  may  jeopardize 
program  success 

Program  critical  path  affected 

Slip  <*month(s) 

Budget  increase  or  unit 
production  cost 
increases. 

<**(l0o/o  of  Budget) 

5 

Severe  degradation  in  technical 
performance.  Cannot  meet  KPP  or 
key  technical/supportability 
threshold;  will  jeopardize  program 
success 

Cannot  meet  key  program  milestones 

Slip  <*month(s) 

Budget  increase  or  unit 
production  cost 
increases. 

>**(10%  of  Budget) 

Consequence 
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5  90%  9 

Requires  new  development  outside  of  any  existing  experience  base.  No 
viable  approaches  exist  that  can  be  pursued  with  any  degree  of  confidence. 
Basic  research  in  key  areas  needed  before  feasible  approaches  can  be 
defined. 

|  ^ 

Requires  new  development  where  similarity  to  existing  experience  base 
can  be  defined  only  in  the  broadest  sense.  Multiple  development  routes 
must  be  pursued. 

Requires  new  development  but  similarity  to  existing  experience  is 
sufficient  to  warrant  comparison  in  only  a  subset  of  critical  areas. 

Multiple  development  routes  must  be  pursued. 

Requires  new  development  but  similarity  to  existing  experience  is 
sufficient  to  warrant  comparison  on  only  a  subset  of  critical  areas.  Dual 
development  approaches  should  be  pursued  in  order  to  achieve  a  moderate 
degree  of  confidence  for  success,  (desired  performance  can  be  achieved  in 
subsequent  block  upgrades  with  high  degree  of  confidence. 

N 

2  30%  4 

3 

1  10%  2 

1 

Requires  new  development  but  similarity  to  existing  experience  is 
sufficient  to  warrant  comparison  in  all  critical  areas.  Dual  development 
approaches  should  be  pursued  to  provide  a  high  degree  of  confidence  for 
success. 

Requires  new  development  but  similarity  to  existing  experience  is 
sufficient  to  warrant  comparison  across  the  board.  A  single  development 
approach  can  be  taken  with  a  high  degree  of  confidence  for  success. 

Requires  new  development  well  within  the  experience  base.  A  single 
development  approach  is  adequate. 

Exists  but  requires  major  modifications.  A  single  development  approach 
is  adequate. 

Exists  with  no  or  only  minor  modifications  being  required.  A  single 
development  approach  is  adequate. 

90% 

80% 

70% 

50% 

40% 

30% 

20% 

10% 

0% 
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5X5  Risk  Matrix 
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•  The  AD2  assessment  provides  the  basis  for  the 

development  of  the  Technology  Development  Plan  and  for 
improved  accuracy  of  the  development  of  program/project 
cost,  schedule  and  risk. 
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